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(54) IMAGE PROCESSING METHOD AND DEVICE 

(57)Abstract: _ 
PROBLEM TO BE SOLVED: ToioBmirf a'pTOtograph* 
\Ajith,ojJt=of=foeus feelingras.if to^'rle'piGkecl up bf ^ ^ 
camera with a large apertujre/rom*1^ 
by. a cameraj^tb^a^stTiliirapertufe letfe. 
SOLUTION: At first a couple of images A, B picked up by 
a couple of image pickup lenses with a prescribed 
parallax simultaneously and parallax information are 
received and a distance distribution of an object group in 
an image is calculated based on the received information 
(S1), an out-of-focus parameter to provide out-of-focus 
information depending on an attribute of the image 
pickup lenses is properly selected (S2), air optical fog - 
different from the actual image pickup lenses is provided 
to the object group depending on the calculated distance^ 

of the object group and the selected out-of-focus parameter (S3), and an image with the fog 
provided thereto is outputted. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3 in the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to the image-processing method and image processing 

system which generate the image which adds dotage to an image and has a dotage taste. 

[0002] 

[Description of the Prior Art] Generally, although the image quality of a film photo has the large place 
depended on the engine performance of a taking lens, since the taking lens of the camera called a high- 
class machine called an one eye reflex camera etc. has large aperture, a bright image can be obtained. 
[0003] And even if it is the case that photographic subject brightness is the same, by choosing various 
combination of drawing and shutter speed, depth of field can be changed, consequently the dotage 
condition of photographic subjects other than a main photographic subject can be controlled freely. That 
an attractive photograph is acquired by the high-class machine has the large place depended on such a 
reason. 
[0004] 

[Problem(s) to be Solved by the Invention] However, the camera with the lens of the above-mentioned 
diameter of macrostomia has the defect that generally the whole camera will weight[ large-sized and ]- 
ize since the taking lens is large-sized. On the other hand, although there is just an advantage that a lens 
shutter camera and a disposable camera are small and that it is lightweight, these taking lenses have 
small aperture, and since it becomes large, it is difficult [ depth of field ] to take a photograph with a 
dotage taste which photoed the lens of the diameter of macrostomia with the camera which it had. 
[0005] This invention was made in view of the above-mentioned problem, and the place made into the 
purpose is to acquire a photograph with a dotage taste which was photoed with the camera equipped 
with the lens of the diameter of macrostomia from the image photoed with the camera equipped with the 
lens with small aperture called a lens shutter camera etc. 
[0006] 

[Means for Solving the Problem] In order to attain the above-mentioned purpose, anJmage-processing 
method by the 1st mode of this invention An^image and the above-mentioned parallax information on a 
pair by which coincidence photography was carried out by passing a taking lens of a pair which has 
predetermined parallax are inputted. Based on information by which the input was carried out [ above- 
mentioned ], dista ^e distribution ofca ohn ^ /V 
com^tga^A^dOgge^parameter forgiving dfij^j^^ of the above - 

^mentioned takingiens is choserTs^ — - 
differei^ above-mentioned ] wiA^istanee 

distributionrof^hotp^^ [ above-mentioned ], 

and which it fades and is depended on an actual taking lens at the above-mentioned photographic subject 
group according to a parameter, and added the above-mentioned dotage. 

[0007] And as for an image-processing me thod by^the 2nd_mode. the above-mentioned parameter is 
characterized by thing pr a local aisiance ot a taldng iensi an <Evalue| and afocus location included for 

(' Q 1 Q 
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feither at least. Furthermore, an^pirt-means^ and the above-mentioned parallax 

information on a pair by which coincidence photography was carried out by an image processing system 
by the 3rd mode passing a taking lens of a pair which has predetermined parallax^A^distance 
distributio n calculation means to compute dista nce distribution of a photogra phic subjegt group in the 
'"above-mentioned image based orTmformafiorTinputted from the above-mentioned input means, Aj 
^ parameter selection me^s to choose s^^^ J ^ 

^ depending on the attr i bute of the TaboveTmentioned l ^ngjl e^pistance distribution of a photographic 
^subfetf group cSifipufed by the above-mentioned di^ncecfistribution calculation means, and a dotage 
r addition means to add different dotage from a thing which was chosen by the above-mentioned 
t parameter selection means and which it fades and is depended on an actual taking lens at the above- 
mentioned photographic subject group according to a parameter, It is characterized by providing an 
image output means to output an image which added dotage with the above-mentioned dotage addition 

[0008] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained with 
reference to a drawing. Drawing 1 is the conceptual diagram of the image-processing method 
concerning the gestalt of operation of this invention. In this drawing, Images A and B are the groups of 
the image which gave parallax and photoed the same photographic subject. Based on the location 
(coordinate) information on each images A and B of each photographic subject, the amount of dotage 
which should be added to each photographic subject from the distance information which computed the 
distance to each photographic subject (step SI), and was computed in this way is computed (step S2), 
dotage is added to Images A and B based on the amount of dotage concerned (step S3), and a generation 
image is obtained. 

[0009] Then, the image processing system which realizes the above-mentioned image-processing 
method is explained in full detail. Drawing 2 is drawing showing the configuration of the camera which 
photos the group of an image which has parallax. In this drawing, two lens 2a and 2bs are arranged by 
the camera 1, and two images which have parallax through concerned lens 2a and 2b are photoed by 
coincidence. If a photographic subject is photoed with a camera 1, a latent image 1 and a latent image 2 
will be exposed by the fields 4a and 4b of a film 3, respectively. A finder 5 is for a user to check the 
photographic subject picturized by field 4a, finally it fades in the image picturized by field 4a, and 
attached processing is given. The film used with a camera 1 is a film with a cartridge of a long volume, a 
film is contained by the cartridge and after development has structure which can insert [ film / 
concerned ] by rotating a spool. 

[0010] Drawing 3 is the external view of the external monitor 200 which displays the dotage additional 
equipment 100 and the processed image for adding dotage to the image picturized with the above- 
mentioned camera 1. The dotage additional equipment 100 becomes from the liquid crystal panel 1 1 
which displays required information in case a user directs the contents of processing in this drawing, the 
directions pad 12 which use in case a user directs and chooses actuation, ****** 13 for opting for 
actuation, the power switch 14 for turning on / turning off a power supply, and insertion opening 15 for 
inserting the film cartridge 10 with which the film developed [ photography ] negatives is contained. 
The dotage additional equipment 100 and the external monitor 200 are connected by the cable 300. 
[001 1] The detailed configuration of the above-mentioned dotage additional equipment 100 is shown in 
drawing 4 , and it explains to it. In this drawing, the lamp 21 for illuminating a film 3 is arranged by the 
dotage additional equipment 100, and the lens 22 is arranged that image formation of the image of the 
film 3 by the lighting of the lamp 21 concerned should be carried out to CCD23. and the image on a film 
3 -- RGB -- it is full color, and CCD23 is arranged in order to picturize. The signal from CCD23 
concerned is connected to this CCD23 in amplification, gamma conversion, and the digital disposal 
circuit 24 for carrying out A/D conversion, and the output of the digital disposal circuit 24 concerned is 
connected to CPU32 which manages the whole control. 

[0012] This CPU32 is equipped with the,storage:section which fades with the control function which 
controls each part, and memorizes a parameter required ^ 
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processing for addition. To this GPU32RAM 25 and 26 which stores the picturized image temporarily, 
and the I/F circuit 27 which changes a picture signal into the signal suitable for a liquid crystal panel 1 1 
or the external monitor 200, The motor 29 made to rotate spool 30 is connected with the operator 
guidance section 28 which consists of a directions pad 12 and ****** 13, and the spool 30 for rolling 
round a film 3. And the gear 3 1 for making the above-mentioned motor 29 and spool 30 engaged is also 
formed, the above-mentioned spool 30 and the spool of a film cartridge 10 are further connected in the 
non-illustrated gear train, and it has composition which is interlocked and rotated. 
[0013] Hereafter, with reference to the flow chart of drawing 5 , actuation of the dotage additional 
equipment 100 in an image processing system is explained to details. If a user inserts the film cartridge 
10 photoed with the camera 1 from the insertion opening 15 and turns on the power switch 14, 
processing will be started, drive control of the motor 29 will be immediately carried out by CPU32, 
spool 30 will rotate, and a film 3 will be pulled out from a film cartridge 10 (step SI 1). 
[0014] It is stopped by the drawer of this film 3 in the exposure region of eye one piece, lighting of a 
lamp 21 and the drive of CCD23 are performed continuously, and the image of eye one piece is 
picturized. Amplification, gamma conversion, and A/D-conversion processing are performed by the 
digital disposal circuit 24, and the picturized picture signal is transmitted to CPU32 after that (step S12). 
A/D conversion of the image here is carried out by RGB8 bits each, and it becomes a total of a 24-bit 
full color image. 

[0015] Then, a picture signal is outputted to the I/F circuit 27 from CPU32, and after being changed into 
the picture signal suitable for a liquid crystal panel 1 1, it is displayed on a liquid crystal panel 1 1 (step 
S 1 3). When anjmage tQ process _by seeing ..the displayed image and a user wishes pushand 
pwcessing-otothercomafop 13, he.chooses ijhe coma Jbr which it v^shes hy.pu^hing the right 

of tKedirections.pad -1 2? 

[0016] When the right of the directions pad 12 is pushed, the (step SI 5 is sent to NO by two pieces in a 
film 3 with branching) and the directions from CPU23, and the image of return (step S14) and the 
following piece is displayed on step S12 by the liquid crystal panel 1 1 . The number of the images by 
which it was picturized with the camera 1 here that a film 3 is sent by two pieces is one in two pieces, 
and they are because it is necessary to send two pieces at a time for displaying the image which photoed 
a different scene. 

[0017] On the other hand, when ****** 13 is pushed, branching), image pick-up, and signal processing 
are performed to YES in the (step SI 5, and a.picture signal is transmitted to RAM25 (step S16). Then, 
the film 3 for one piece is sent (step SI 7), image pick-up and signal processing are performed, and a 
picture signal is transmitted to RAM26 (step SI 8). Here, the images memorized by RAM26 are the 
image memorized by RAM25 and an image photoed with parallax at this time of day, and they are used 
in order to compute the distance of a photographic subject behind. 

[0018] Then, it moves from processing to dotage attached processing. That is, a "setting item" menu like 
drawing 6 is first displayed on a liquid crystal panel 1 1 . A user can choose an item to set up by pushing 
the upper and lower sides^ ^ it can go into the sub menu of each setting item 

by pushing ****** 13. After finishing each setup, it fades by choosing "setting termination" and the 
"parameter setup" of attached processing is ended (step S19). About the setting item to which a user did 
not set, a default like drawing 6 is used as a parameter. In addition, a "parameter setup" is explained in 
full detail behind. 

[0019] In this way, "dotage attached processing" is performed by CPU32 using the set-up parameter 
(step S20), after a processing image is changed into the picture signal which was outputted to the I/F 
circuit 27 and was suitable for the external monitor 200 from CPU32, it is displayed on the external 
monitor 200 (step S21), and this sequence is ended. 

[0020] Next, with reference to drawing 6 and drawing 7 , the above "a parameter setup" is explained in 
full detail. After choosing the piece which a user wants to process and picturizing the target image, a 
"setting item" menu as shown in drawing 6 (a) is displayed on a liquid crystal panel 11. 
[0021] In this drawing, the item surrounded by the frame shows the item by which current selection is 
made. A change of selections is made by carrying out the depression of the upper and lower sides of the 
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directions pad 12, and the decision of selections is made by carrying out the depression of ****** 13. 
By "a lens class setup 1 ' in a^setting item", the class of taking lens which a user wants to use, i.e., the 
taking lens of imagination, is chosen. Since the field angle photoed with a taking lens and the magnitude 
of dotage change, the image at the time of taking a photograph with a different lens from the lens 
actuallyjised for photography is generable by processing in consideration of these. 
[0022] If "a lens class setup" is actually chosen, a "lens class setting" sub menu as shown in drawing 6 
(b) will be displayed. The default of a "lens class" is "100mmF2", and " 100mmF2" is surrounded by the 
frame immediately after the sub menu display. Ifmodification and decision of a lens which a user 
chooses are made, ? "setting item" menu will be displayed again. 

[0023] And the range which should be processed in the whole image is set up in a "processing entry." As 
shown in drawing 7 (a), it can-choose now from four kinds of ranges of A-D here. These A-D is equal to 
thelielH^gSat the time of taking a photograph with a lens (50mm, 100mm, 200mm, and 300mm), 2 
T ! e^gSvefy7and supports four kinds of lenses set up by "lens class setup." Drawing 7 (a) is instantiation 
of the processing range at the time of choosing B. Selection of a "processing entry" displays the image 
memorized by RAMI on a liquid crystal panel 11. The default of a "processing entry" is B and the 
processing range B is expressed as the continuous line frame immediately after image display. 
Moreover, about the ranges of other, it is expressed as the dotted line frame, if a user makes 
modification and the decision of a processing range, a "setting item" menu will be displayed again? 
[0024] Then, in "an f number setup", the f number of a taking lens, i.e., the taking lens of imagination, to 
use at the time of photography is set up. Since the dotage condition of the image photoed by the f 
number at the time of photography changes, the image at the time of taking a photograph by the 
different f number from the f number at the time of actual photography is generable by fading in 
consideration of these and giving attached processing. Selectidfrof- an f number setup" displays an "f 
number" sub menu as shown in drawing 6 (c). The default of the "f number" is "2.8" and "2.8" is 
surrounded by the frame immediately after the sub menu display. If modification and the decision of the 
f number which a user chooses are made, a "setting item" menu will be displayed again. 
[0025] And the location of a photographic subject to double a focus is set up by "it is a fopus location 
just." There are two kinds-of setting methods of of "a distance setup" which sets up a focus location just 
with the distance to a photographic subject to double a focus from a photography person, and the 
"location" which sets up a focus location just by specifying the location of the photographic subject in 
an image in this setup. Selection of "a distance setups displays a J, distance setting" sub menu, as shown 
in drawing^ (e). The default of a "distance setup" is "5m" and "5m" is surrounded by the frame 
immediatelyVter the sub menu display. If modification of the "distance" which a user chooses, and a 
decision are made, a "setting item" menu will be displayed again. On the other hand, selection of a 
"location" displays the processing range set up by the above "a lens class setup" or a "processing entry" 
on the above-mentioned liquid crystal panel 1 1 . 

[0026] Drawing 7 (b) is the example of the image displayed when "a lens class setup" is shown and "B" 
is chosen by "100mmF2" or a "processing entry." A focus location is just specified by moving 
BOINTINGU cursor as shown in this drawing 7 (b) with the directions pad 12, and pushing ****** 13. 
This drawing 7 (b) is the example which set a person's frame as the focus location just. Termination of 
the above setup displays a "setting item" menu again. 

[0027] The above parameter setup is completed by choosing "setting termination" in the above- 
mentioned "a setting item" menu. When the set-up parameters are enumerated, it is a lens class (default 
100mmF2). 

Processing range (default B) 

F number (default F2.8) 

It is a focus location (default 5m) just. 

It becomes. 

[0028] Next, "dotage attached processing" is explained in full detail. As shown in drawing 8 , the image 
pick-up range at the time of picturizing the image on a film 3 by CCD23 is set up so that perforation 3a 
and latent-image 4a which are located in the upper left of an image may be included. The coordinate 
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location of the pixel equivalent to the lower left corner of the perforation of the image (image A) 
containing this perforation is made into Point O, and the coordinate location of the point P of the 
arbitration on an image is expressed with Vector OP on the basis of Point O. In case this computes 
photographic subject distance behind, it is because the amount of gaps of the location of each 
photographic subject picturized by the location where it differs on two parallax images is needed. 
[0029] Since a certain criteria location is needed in order to decide the absolute location where the 
photographic subject was picturized, the relative position from there defines the absolute location of 
each photographic subject here by making the lower left corner of the perforation of a film 3 into a 
criteria location. If the pixel location of the upper left corner of an image is made into point 0\ it is 
decided that relative-position vector 00' from perforation 3a will be the structure top constant value of a 
camera 1. Therefore, by lengthening vector 00' from Vector OP, vector O'P is calculated and it can use 
as an absolute position coordinate of Point P. 

[0030] It is Image B which started only the film image portion of Image A by making this point O' into a 
zero. Image B is dispersed to a horizontal N pixel and M pixels long, and expresses the pixel location of 
width and each length with i and j. Furthermore, the image which divided width into n pieces and 
divided length into m fields for Image B is used as Image C, and the divided field is set to R (I, J). It 
prepares for every parallax image of two sheets which photoed such an image C to coincidence, and let 
an image [ as opposed to an image CI and latent-image 4b for the image to latent-image 4a ] be an 
image C2. 

[0031] Next, the procedure which computes the distance of a photographic subject from images CI and 
C2 is explained. The field R2 (I, J) on the image C2 with which the photographic subject which is 
equivalent to the field Rl (10 and JO) on an image CI first is picturized is computed. Here, correlation 
with Rl (10 and JO) and R2 (I, J) is calculated, and let the field where correlation is the highest be the 
correspondence field of Rl (10 and JO). 

[0032] That is, correlation COL (I, J) is set to COL(I, J) =sigmasigma(Pl-(i, j) P2)-(k, I) 2. 
[0033] Here, PI and (i, j) P2 (k, 1) show the value of G of the pixel contained in Rl and (10 and JO) R2 
(I, J), respectively. It is because space resolution of G is [ using the value of G for count of a correlation 
value ] the highest in RGB and the precision of a correlation value is good. This correlation value is 
computed by continuing throughout an image, and it asks for I and J to which COL (I, J) becomes the 
largest, them -- respectively - II and Jl ** -- if it carries out - a field R2 (II and Jl) - Rl (10 and JO) 
and abbreviation - it is the field where the same photographic subject is picturized. When width of face 
of field 4a is set to 36mm, the resolution RSL of an image CI and an image C2 is RSL=N/36 
[pixel/mm]. 
It becomes. 

[0034] Here, drawing 9 is drawing showing the physical relationship of the photographic subject 1 when 
photoing a photographic subject 1, and a camera 1. When the opening of lens 2a and lens 2b is 
sufficiently small, the image of a photographic subject is reflected as **** on a film 3. The distance of a 
photographic subject is computable from the physical relationship of Rl (10 and JO) and R2 (II and Jl) 
which were calculated previously. If distance from lens 2a and lens 2b to a film 3 is set to f, the distance 
L from lens 2a and lens 2b to a photographic subject will be found by the degree type. 
[0035] When, and it is set to L= 2500mm, that is, this image is picturized, it turns out that the distance 
from a camera to a photographic subject was 2.5m. [ L=a*f*RSL/|I0-Il |, a=50[mm] f=50[mm] 
RSL=100[pixel/mm] 10 =500 / for example, /, and ] [ II =] the above count - each ** Rl of an image 
CI ******** „ by carrying out repeatedly, distance distribution of the photographic subject currently 
picturized by the image CI can be found. 

[0036] Next, the procedure which fades from photographic subject distance, judges an amount, and adds 
dotage is explained. Since an image CI is a digital image, dotage is added by collapsing a matrix in each 
pixel of an image. Twoior^more-matriGes which correspond to CPUS^^at the point spread function 
(PSF;point spread fanction ) of -a lens are memorized at tl^time of manufacture. In addition, this PSF is> 
a function with which **** of a photographic subject shows how it spreads according to a focus 
condition, and shows the optical reinforcement (brightness) in a film plane up coordinate. Dotage is 
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added by choosing and collapsing a suitable matrix in each field R of every (I, J) according to the 
parameter set up by the above-mentioned "parameter^setup", and the already„computed photographic 



[0037] As a parameter which chooses a matrix, there are four, "the:focal distance of a lens", the "f> 
humbfer", "it beingJocus:distance,just n , and "photographic subject distance." The focal distance of a lens 
is judged from the 'lens class" set up by the "parameter setup", and a^processihg range." Focus distance 
uses just the value which changed into "distance" the "location" specified using the photographic subject 
distance which already calculated the value as it is when "a distance setup" is performed by "it is a focus 
location just" when a "location" was performed. 

[0038] If the focakdistance and the f number of a lens are decided to be a certain value, the magnitude of 
dotage will carry out proportionally [ abbreviation ] just from the property of a lens at the difference of 
the inverse number of focusRiistance, and the inverse number of photographic subject distance. The 
number of parameters can be reduced without dropping the precision of dotage addition from this by 
using the difference of the inverse number of focus distance, and the inverse number of photographic 
subject distance as a new parameter just. This parameter is made into "range difference." 
[0039] These parameters are numbered according to the table of drawing 10 , and the dotage matrix 
which has these as an argument is defined as follows. 

Mf, F, D:(k, 1) f(focal distance) =1, 2 and 3, 4F(f number) =1, 2, 3, 4, 5, 6, 7, 8D (range difference) = the 
number of the possible combination of 1, 2, 3, 4, 5, 6f, and F and D to a matrix is Ml -M156. 156 kinds 
are prepared. Although the range of k and 1 is decided by magnitude of the dotage matrix M, the matrix 
of 5x5 of -2<k<2 and -2<1<2 is used here. The element of each matrix is computed from PSF of the lens 
which should reproduce dotage, and a value changes with each parameter. Since f and F are decided as a 
parameter peculiar to an image CI, the parameter which changes the matrix made to act according to the 
location in an image is set only to D. 

[0040] Here, the example of the matrix in the case of a default (f= 2, F= 3) is shown in drawing 1 1 . It is 
because computation time can be sharply shortened compared with the method of having chosen the 
suitable matrix according to the parameter from two or more such matrices fading for each point of an 
image, and computing a matrix. Thus, pixel value P' which added dotage when the selected matrix was 
used is set to P'(i, j) =sigmasigmaMf, F, D and (k, 1) P(i-k, j-l)/sigmasigmaMf, and F and D (k, I). 
[0041] By repeating such count about each point of an image CI, and performing it, the dotage 
according to photographic subject distance can be added to an image CI . 
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CLAIMS 



[Claim(s)] 

[Claim 1] An image and the above-mentioned parallax information on a pair by which coincidence 
photography was carried out by passing a taking lens of a pair which has predetermined parallax are 
inputted. Distance distribution of a photographic subject group by which computed distance distribution 
of a photographic subject group in the above-mentioned image based on information by which the input 
was carried out [ above-mentioned ], and chose suitably a dotage parameter for giving dotage 
information depending on the attribute of the above-mentioned talcing lens, and calculation was carried 
out [ above-mentioned ], An image-processing method characterized by outputting an image which 
added different dotage from a thing by which selection was made [ above-mentioned ], and which it 
fades and is depended on an actual taking lens at the above-mentioned photographic subject group 
according to a parameter, and added the above-mentioned dotage. 

[Claim 2] The above-mentioned parameter is the image-processing method according to claim 1 
characterized by thing of a focal distance of a taking lens, an F value, and a focus location included for 
either at least. 

[Claim 3] An image processing system characterized by providing the following An input means to 
input an image and the above-mentioned parallax information on a pair by which coincidence 
photography was carried out by passing a taking lens of a pair which has predetermined parallax A 
distance distribution calculation means to compute distance distribution of a photographic subject group 
in the above-mentioned image based on information inputted from the above-mentioned input means A 
parameter selection means to choose suitably a dotage parameter for giving dotage information 
depending on the attribute of the above-mentioned taking lens Distance distribution of a photographic 
subject group computed by the above-mentioned distance distribution calculation means, a dotage 
addition means add different dotage from a thing which was chosen by the above-mentioned parameter 
selection means and which it fades and is depended on an actual taking lens at the above-mentioned 
photographic subject group according to a parameter, and an image output means output an image which 
added dotage with the above-mentioned dotage addition means 



[Translation done.] 
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